r 




START 
MAIN PROGRAM 



CREATE PARENT WINDOW 



DRAW BUTTONS ON 
PARENT WINDOW 



FIG. 3C 



CREATE SAGITTAL CHILD 
WINDOW 



I 



CREATE A/P 
CHILD WINDOW 



IF A 
BUTTON OR A 
KEY HAS BEEN PRESSED^ 
OR THE MOUSE 
HAS MOVEQ 



ELSE 



122 



I THEN 



IF 

"^ACQUIRE A/P IMAGE^ 
OR F1 KEY 

THEN 

IF 

"ACQUIRE 
SAGITTAL IMAGE" OR 
F2 KEY 

THEN 

IF 

"SELECT A/P 
FIDUCIALS" OR 
F3 KEY 



THEN 



ELSE 



^TO BLOCK 166 



ELSE 



-TO BLOCK 200 



ELSE 



TO BLOCK 234 



FROM FIGURE 3F 



TO FIGURE 3D 




THEN 



TO FIG. 3E 



FROM FIG. 3D 



FIG. 3E 



IF 

"SAGIHAL 
ANGLE" OR 
F10KEY 



ELSE 



TO BLOCK 41 2 



THEN 



IF 

'ROBOT MOVE' 
OR F11 KEY 



ELSE 



TO BLOCK 422 



THEN 

IF 

"MOVE 
ALONG DRILL AXIS" 
OR F12 KEY. 



ELSE 



*■ TO BLOCK 452 



THEN 



ELSE 



TO BLOCK 476 



THEN 



IF 

TRACKBALL 
OVER THE "SAGIHAL: 
IMAGE area: 



ELSE 



^TO BLOCK 506 



IF 

'TRACKBALL HAS^ 
BEEN CLICKED OVER 
THE "ROBOT 
;ONTROLAREA' 



ELSE 



-TO BLOCK 536 



THEN 



1 1 

TO FIG. 3F 



FIG. 3F 



154 



156 



FROM FIG. 3E 



THEN 



FREE ALL ALLOCATED 
MEMORY 



TO FIG. 3C 
i 




ELSE 




166 



< 



ACQUIRE A/P IMAGE BUTTON 
OR F1 KEY HAS BEEN PRESSED 



ELSE 



174 



Ml, 




FIG. 4 




178 



180 



DISPLAY A/P 
IMAGE 



DISABLE & DIM 
BUTT0NS:F11,F12, F5 
ENABLE & 
BRIGHTEN 
BUTTONS: F3 



ASSIGN THE NULL 
END-EFFECTOR 



SET PASSTHRU MODE — 170 



^^^^—172 



••-c--; 



188 



182 



Tfthi 

"END-EFFECTOft 
.GETS ASSIGNS 
OK 

jTHEN 



ELSE 



GET CURRENT POSITION OF 
ROBOT-END-EFFECTOR FROM 
ROBOT CONTROLLER 




ERROR: . 
"END-EFFECTOR 
NOT ASSIGNED!" 



|rCTURnJ — 186 



184 



190- 



IF 

Sagittal imag^ 
is displayed. 

THEN 



ELSE 



SEND MESSAGE: 
"ACQUIRE SAGITTAL IMAGE" 



192 



196 



SEND MESSAGE: 
"SELECT THE FIDUCIALS." 



198 — ^E^RN^ 




200 



ACQUIRE SAGITTAL IMAGE BUTTON 
OR F2 KEY HAS BEEN PRESSED 



202 



ELSE 



208 




FIG. 5A 



SET PASSTHRU MODE 



204 



210 




212 



214 



DISPLAY 
SAGITTAL 
IMAGE 

~1 



DISABLE KEYS: 

F11, F12. F6 
ENABLE KEYS: 
F4 

DIM BUTTONS: 
"MOVE ROBOT". "MOVE 

ALONG AXIS", AND 
"REGISTER SAGITTAL 

IMAGE" 
BRIGHTEN BUTTONS: 
"SELECT SAGITTAL 
FIDUCIALS." 



ASSIGN THE NULL 
END-EFFECTOR 



^^^^--206 



216 



222 



IF THE 
END-EFFECTOR' 

lETS ASSIGNEI 
OK 

THEN 



ELSE 



ERROR: 
"END-EFFECTOR 
NOT ASSIGNED!" 



GET CURRENT POSITION OF 
ROBOT-END-EFFECTOR FROM 
ROBOT CONTROLLER 



^RETU^) — 220 



TO FIGURE 5B 



FIG. 5B 



FROM FIGURE 5A 



224 



IF 

A/P IMAGE 
IS DISPLAYED 

THEN 



ELSE 



230 



SEND MESSAGE: 
"SELECT THE FIDUCIALS." 



~ I ft 

3 . => 



232 —(return^ 



SEND MESSAGE: 
"ACQUIRE A/P IMAGE' 



(rCTURn)— 228 



226 



234 



SELECT A/P FIDUCIALS 
BUTTON OR F3 KEY HAS 
BEEN PRESSED 



FIG. 6A 



236 



242 




ELSE 



ERROR: 
'ACQUIRE A/P IMAGE" 



/display SQUARE 
\ CURSOR j 



(return)— 240 



238 



244 



RESET NUMBER OF 
FIDUCIALS TO ZERO 



248 




250 



PERFORM EDGE DETECTION 
AROUND THE MOUSE COORDINATE. 
EDGE DETECTION IS PERFORMED 
USING J. CANNrS [1] GRADIENT 
BASED METHOD 




254 
I 



> ERROR: 
"TRY AGAIN CLOSER 
TO THE FIDUCIAL- 



THEN 



256 



MAP THE EDGE PIXELS TO 
THEIR CALIBRATED IMAGE 
COORDINATES [13]. 



TO FIGURE 6B 



FROM FIGURE 6B 



258 



262 



FROM FIG. 6A 



i 



FIND THE CENTER OF THE 
FIDUCIAL SHADOW USING THE 
CALIBRATED EDGE PIXELS [14] 



T 



DRAW A CIRCLE AROUND 
THE CENTER OF THE 
FIDUCIAL SHADOW 



264 



IF 

ALL EIGHT 
f IDUCALS HAVE BEEI 
LOCATED 



ELSE 



268 



270 



272 



274 



THEN 



SAVE THE ESTABLISHED 
IMAGE COORDINATES 
OF ALL FIDUCALS 



I 



ENABLE KEY: F5 
BRIGHTEN BUTTON: 
"REGISTER A/P IMAGE" 



SEND MESSAGE: 
"REGISTER A/P IMAGE" 



GOTO: 
ENTRY1 IN FIGURE 8 



276 



r SELECT SAGITTAL FIDUCIALS 
BUTTON OR F4 KEY HAS 
BEEN PRESSED 



FIG. 7A 




ELSE 



ERROR: 
'ACQUIRE SAGITTAL IMAGE" 



(return) — 282 



280 



292 



RESET NUMBER OF 
FIDUCIALS TO ZERO 



296 




298 



PERFORM EDGE DETECTION 
AROUND THE MOUSE COORDINATE. 
EDGE DETECTION IS PERFORMED 
USING J. CANNY'S [1] GRADIENT 
BASED METHOD 



300 



IF AT 
LEAST 3 EDGE 
PIXELS FOUm 

THEN 



302 
I 



ELSE 



ERROR: 
"TRY AGAIN CLOSER 
TO THE FIDUCIAL" 



304 



MAP THE EDGE PIXELS TO 
THEIR CALIBRATED IMAGE 
COORDINATES [13], 

' — r 

TO FIGURE 7B 



FROM FIGURE 7B 



306 



310 



FROM FIG. 7A 



1 



FIND THE CENTER OF THE 
FIDUCIAL SHADOW USING THE 
CALIBRATED EDGE PIXELS [14] 



DRAW A CIRCLE AROUND 
THE CENTER OF THE 
FIDUCIAL SHADOW 



312 



IF 

ALL EIGHT 
f IDUCALS HAVE BEEl 
LOCATED 



ELSE 



316 



318 



320 



322 



THEN 



SAVE THE ESTABLISHED 
IMAGE COORDINATES 
OF ALL FIDUCALS 



I 



ENABLE KEY: F6 
BRIGHTEN BUTTON: 
"REGISTER SAGITTAL IMAGE" 



SEND MESSAGE: 
"REGISTER SAGITTAL 
IMAGE" 



GOTO: 
ENTRY2 IN FIGURE 9 



324 



_r REGISTER A/P IMAGE BUHON^ 
~\^0R F5 KEY HAS BEEN PRESSED J 



FIG. 8 



326 



IF ALL A/P 
FIDUCIALS HAVE 
BEEN FOUND. 



ELSE 



ENTRY1: 



4THEN 



332 



334 



336 



338 



RECALL 2D COORDINATES 
OF THE A/P FIDUCIAL CENTERS. 



READ IN FROM FILE THE 
3D CMMED COORDINATES OF 
THE CENTER OF THE FIDUCIALS. 



OPTIMIZE THE FIT BETWEEN THE 
3D CMMED COORDINTES AND THE 
2D IMAGE COORDINATES USING 
LEVENBERG-MARQUARDT'S 
METHOD. [2] 



CONSTRUCT REGISTRATION 
MATRIX [3] 



ERROR: "HAVEN'T 
SELECTED ALL THE 
FIDUCIALS" 




330 



340 



346 



IF 

SAGITTAL 
IMAGE HAS BEEN 
REGISTERED. 

THEN 



SEND MESSAGE: 
"PICK ENTRY POINT" 



ELSE 



SEND MESSAGE: "PERFORM 
SAGITTAL REGISTRATION" 




344 




/REGISTER SAGITTAL IMAGE BUHOfh 
OR F6 KEY HAS BEEN PRESSED J 



FIG. 9 




352 



Tf all SAGinAC 

FIDUCIALS HAVE 
BEEN FOUND. 



ELSE 



ENTRY2: 



4THEN 



358 



RECALL 2D COORDINATES 
OF THE SAGIHAL FIDUCIAL CENTERS. 



360 



READ IN FROM FILE THE 
3D CMMED COORDINATES OF 
'THE CENTER OF THE FIDUCIALS, 



I 



362 



OPTIMIZE THE FIT BETWEEN THE 
3D CMMED COORDINTES AND THE 
2D IMAGE COORDINATES USING 
LEVENBERG-MARQUARDTS 
METHOD. [2] 



364 



CONSTRUCT REGISTRATION 
MATRIX [4] 



366 




ELSE 



THEN 



372 



SEND MESSAGE: 
TICK ENTRY POINT" 



ERROR: "HAVENT 
SELECTED ALL THE 
FIDUCIALS" 




356 



SEND MESSAGE: "PERFORM 
■ A/P REGISTRATION" 




370 




376 



/ TRANSVERSE ANGLE BUTTON \ 



yOR F7 KEY HAS BEEN PRESSED^ 



378 



PLACE CURSOR IN 
THE ENTRY FIELD 



380 



ENTER THE NUMERIC VALUE 
FOR THE TRANSVERSE 

ANGLE 



I 



382 



READ IN (3 AND UPDATE THE 
ORIENTATION OF THE 
VIRTUAL GUIDEWIRE 



I 



384 



REDRAW THE 
VIRTUAL GUIDEWIRE 
IN BOTH THE A/P IMAGE 
AND SAGITTAL 
IMAGE [7] 






388 



390 



392 



394 



SCREW LENGTH BUTTON 
OR F8 KEY HAS BEEN PRESSED 



I 




PLACE CURSOR IN 
THE ENTRY FIELD 



ENTER THE NUMERIC VALUE 
FOR THE SCREW LENGTH 



I 



READ IN SCREW LENGTH AND 
UPDATE LENGTH OF THE 
VIRTUAL GUIDEWIRE [11]. 



396 



REDRAW THE 
VIRTUAL GUIDEWIRE 
IN BOTH THE A/P IMAGE 
AND SAGITTAL 
IMAGE [7]. 



398 





FIG. 11 



400 



402 



404 



406 



408 



SAGITTAL ANGLE BUTTON ^ 
.OR F10 KEY HAS BEEN PRESSEPy 



I 



PLACE CURSOR IN 
THE ENTRY FIELD 



ENTER THE NUMERIC VALUE 
FOR THE SCREW LENGTH 



I 



READ IN cx AND UPDATE THE 

ORIENTATION OF THE 
• VIRTUAL GUIDEWIRE [10]. 



I 



REDRAW THE 
VIRTUAL GUIDEWIRE 
IN BOTH THE A/P IMAGE 
AND SAGITTAL 
IMAGE [7] 




410 




FIG. 12 



r APPROACH ANGLE BUTTON ^ 
yOR F9 KEY HAS BEEN PRESSED^ 



FIG. 13 



414 



PLACE CURSOR IN 
THE ENTRY FIELD 



416 



3 :;• 



ENTER THE NUMERIC VALUE 
FOR THE APPROACH ANGLE 



418 



READ IN Y 



422 



424 



426 



428 



430 



3 



434 



440 



442 



444 



MOVE ROBOT BUTTON OR 
F11 KEY HAS BEEN PRESSED 



I 



FIG. 14 



RECALL APPROACH 
ANGLE, Y 



RECALL a, p, TX. TY. TZ 



CALCULATE THE PLANNED 
POSITION AND ORIENTATION 

[12] 




READ IN FROM FILE THE 
SURGICAL END-EFFECTOR 
DESCRIPTION AND ASSIGN IT. 




, ELSE J 


ERROR: 
"SURGICAL END-EFFECTOR 
NOT ASSIGNED" 















436 



THEN 



SEND ROBOT COMMAND TO 

MOVE TO THE PLANNED 
POSITION AND ORIENTATION 




ASSIGN THE NULL 
END-EFFECTOR 




ELSE 



ERROR: 
"NULL END-EFFECTOR 
NOT ASSIGNED" 



446 
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